Information Extraction and Passport Recognition

Abstract

The passport recognition and information extraction project aims to develop a software system that can automatically recognize and extract key information from passports. The system will utilize computer vision techniques to process passport images and extract important data such as the passport holder\'s name, date of birth, passport number, and expiration date. The extracted information will then be used to populate a database or other software systems for further processing. The project will involve the use of deep learning algorithms for image recognition and feature extraction, as well as natural language processing techniques for text extraction and analysis. The system will be designed to operate accurately and efficiently across a range of different passport types and will be suitable for use in a variety of applications, including border control, identity verification, and travel management.

Introduction

I. INTRODUCTION

Passport recognition and information extraction is a critical task in today's world, where the need for secure and efficient travel management and identity verification is growing rapidly. The process of manually verifying passport information can be time-consuming, error-prone, and costly, making it essential to develop automated systems that can accurately recognize and extract passport data. The passport recognition and information extraction aims to develop a software system that utilizes cutting-edge computer vision and natural language processing techniques to automatically recognize and extract key information from passport images. The system will leverage deep learning algorithms for image processing and feature extraction and will be designed to operate efficiently and accurately across a range of different passport types.

II. LITERATURE REVIEW

This section provides a brief overview of the existing work carried out in the field of text recognition. Text recognition has been in existence since a very long time.

In [1], A Survey" by Gaurav Sharma, et al. (2020) is a survey paper that provides a comprehensive review of the latest research on passport recognition. Passport recognition refers to the process of automatically extracting information from passports, such as personal details, expiration date, and issuing authority. The paper then reviews the different techniques and algorithms used in passport recognition, such as optical character recognition (OCR), machine learning, and deep learning. It provides a detailed description of each technique and highlights their strengths and weaknesses. Finally, the paper concludes by highlighting future directions for research in passport recognition, such as the development of more robust and efficient algorithms, the integration of new technologies, and the exploration of new applications.

In [2], A Comprehensive Survey" by Ali M. Al-Salman, et al. (2021) is a survey paper that provides a comprehensive overview of the latest research on information extraction techniques from unstructured text. Unstructured text refers to text data that is not organized in a predefined format, such as social media posts, emails, and news articles. The paper begins by discussing the importance of information extraction in various applications, such as sentiment analysis, customer relationship management, and social media monitoring. It then provides an overview of the different techniques used for information extraction, such as natural language processing (NLP), machine learning (ML), and deep learning (DL). Finally, the paper concludes by highlighting future directions for research in information extraction, such as the development of more accurate and efficient algorithms, the exploration of new applications, and the integration of new technologies, such as knowledge graphs and semantic web.

In [3], A Survey" by Omar Alobaidi, et al. (2021) is a survey paper that provides a comprehensive review of the latest research on automatic information extraction from identity documents, such as passports, national ID cards, and driver's licenses. The paper then reviews the different techniques and algorithms used for automatic information extraction from identity documents, such as optical character recognition (OCR), machine learning, and deep learning. It provides a detailed description of each technique and highlights their strengths and weaknesses.

Finally, the paper concludes by highlighting future directions for research in automatic information extraction from identity documents, such as the development of more accurate and efficient algorithms, the exploration of new applications, and the integration of new technologies, such as blockchain and biometrics.

In [4], Konstantin Dorofeev, et al. (2022) is a survey paper that provides a comprehensive review of the latest research on deep learning techniques for information extraction. Information extraction refers to the process of automatically extracting structured information from unstructured data sources such as text and images. The paper then reviews the different applications of deep learning techniques for information extraction, such as named entity recognition, relation extraction, and event extraction. Finally, the paper concludes by highlighting future directions for research in deep learning-based information extraction, such as the development of more accurate and efficient algorithms, the exploration of new applications, and the integration of new techniques, such as multi-task learning and reinforcement learning.

In [5], Meenakshi V. Balaji, et al. (2022) is a survey paper that provides a comprehensive overview of the latest research on passport recognition techniques and their applications. The paper begins by discussing the importance of passport recognition in various applications, such as border control, airport security, and law enforcement. Furthermore, the paper discusses the different applications of passport recognition, such as passport verification, identity verification, and border control. It provides a detailed description of each application and highlights their potential impact. Finally, the paper concludes by highlighting future directions for research in passport recognition, such as the development of more accurate and efficient algorithms, the exploration of new applications, and the integration of new technologies, such as biometrics and blockchain.

In [6], This paper by Xiaoning Song et al. proposes a deep learning approach for passport recognition that uses convolutional neural networks (CNNs) to extract features from passport images and achieve high accuracy. The authors first discuss the importance of passport recognition in various applications, such as border control and airport security, and then highlight the challenges in recognizing passports due to their complex and diverse features.  This paper demonstrates the effectiveness of using CNNs for passport recognition and provides insights into the potential of deep learning techniques in image-based document recognition applications.

In [7], This paper By Jianhui Zhao et al. proposes a transfer learning approach for passport recognition that utilizes pre-trained deep convolutional neural networks (CNNs) to extract features from passport images and achieve high accuracy. Overall, this paper demonstrates the effectiveness of using transfer learning with pre-trained CNNs for passport recognition and provides insights into the potential of utilizing pre-trained models for document recognition applications.

In [8], This paper by Rong Chen et al. proposes a machine learning-based approach for information extraction from medical reports that uses natural language processing (NLP) techniques to achieve high accuracy. Overall, this paper demonstrates the effectiveness of using NLP and machine learning techniques for information extraction from medical reports and highlights the potential of these techniques in other applications where structured information is needed from unstructured text data.

In [9], This paper  by Amir Hossein Jafari et al. proposes a framework for automatic information extraction from business cards using a combination of deep learning and rule-based methods to achieve high accuracy. The proposed approach involves several steps, including image preprocessing, text detection, text recognition, and information extraction. The authors use a combination of deep learning-based methods, such as convolutional neural networks (CNNs) for text detection and recognition, and rule-based methods, such as regular expressions and heuristics, for information extraction. Overall, this paper demonstrates the effectiveness of using a combination of deep learning and rule-based methods for information extraction from business cards and highlights the potential of these techniques in other applications where structured information is needed from unstructured text data.

In [10], This paper " by Xiaoping Song et al. proposes a deep learning-based approach for passport recognition that uses multi-view feature fusion to improve recognition accuracy. The authors first describe the challenges in passport recognition, such as variations in passport layout, image quality, and illumination conditions. They then explain the concept of deep learning, which involves using neural networks to automatically learn and extract features from data. To evaluate their approach, the authors use a dataset of passport images and compare their results with other state-of-the-art methods. The proposed approach achieves high accuracy in recognizing different types of passports and outperforms other methods in terms of recognition rate.

III. METHODOLOGY

In this section we briefly describe the overall architecture of text recognition system. A text recognition system receives an input in the form of image which contains some text information. The output of this system is in electronic format i.e. text information in image are stored in computer readable form. The text recognition system can be divided in following modules:

A. Pre-processing Module

The paper document is generally scanned by the optical scanner and is converted in to the form of a picture. A picture is the combinations of picture elements which are also known as pixels. At this stage we have the data in the form of image and this image can be further analyzed so that’s the important information can be retrieved. So to improve quality of the input image, few operation are performed for enhancement of image such as noise removal, normalization, binarization etc.

1. Noise Removal

Noise removal is one of the most important process. Due to this quality of the image will increase and it will effect recognition process for better text recognition in images. And it results in generation of more accurate output at the end of text recognition processing. There are many methods for image noise removal such as mean filter, min-max filter, Gaussian filter etc.

2. Normalization

Normalization is one of the important pre-processing operation for text recognition. The normalization is applied to obtain characters of uniform size, slant and rotation.

3. Binarization

Binarization is one of the important pre-processing operation for text recognition. A printed document is first scanned and is converted into a gray scale image. Binarization is a technique by which the gray scale images are converted to binary images. This separation of text from background that is required for some operations such as segmentation

B. Text Recognition Module

This module can be used for text recognition in output image of pre-processing model and give output data which are in computer understandable form. Hence in this module following techniques are used.

1. Segmentation

In text recognition module, the segmentation is the most important process. Segmentation is done to make the separation between the individual characters of an image.

2. Feature Extraction

Feature extraction is the process to retrieve the most important data from the raw data. The most important data means that’s on the basis of that’s the characters can be represented accurately. To store the different features of a character, the different classes are made. There are many technique used for feature extraction like Principle Component Analysis (PCA), Linear Discriminate Analysis (LDA), Independent Component Analysis (ICA), Chain Code (CC), zoning, Gradient Based features, Histogram etc.

3 Classification 

The classification is the process of identifying each character and assigning to it the correct character class, so that texts in images are converted in to computer understandable form. This process used extracted feature of text image for classification i.e. input to this stage is output of the feature extraction process. Classifiers compare the input feature with stored pattern and find out best matching class for input. There are many technique used for classification such as Artificial Neural Network (ANN), Template Matching, Support Vector Matching (SVM) etc.

C. Post-Processing Module

The output of text recognition module is in the form text data which is understand by computer, so there need to store it in to some proper format (i.e., text or MS-Word) for farther use such as editing or searching in that data.

To develop a passport recognition and information extraction system using machine learning, the following components are needed:

1. Dataset: A large dataset of passport images is needed for training the machine learning algorithm. The dataset should include a variety of passport images from different countries, with different backgrounds, lighting conditions, and orientations.
2. Training Data: The passport images in the dataset should be labeled with the relevant information that needs to be extracted from them, such as the passport holder's name, date of birth, passport number, etc.
3. Machine Learning Algorithm: A suitable machine learning algorithm needs to be chosen, such as Convolutional Neural Networks (CNN) or Recurrent Neural Networks (RNN). These algorithms can be used to extract features from the passport images and classify them based on their content.
4. Input Data: The passport images that need to be recognized and information extracted from them are the input data.
5. New Data: New passport images that were not included in the original dataset will be used to test the accuracy of the machine learning model.
6. Classifier: The classifier will use the extracted features from the passport images to classify them into different categories based on the information they contain.
7. Prediction: The machine learning algorithm will make a prediction on the extracted information from the passport image.
8. Result: The result will be the extracted information from the passport image, which can be further used for processing or verification.

Overall, developing a passport recognition and information extraction system requires a complex process of data collection, preprocessing, training, and testing of the machine learning algorithm. However, once a suitable algorithm is developed, it can be used to process a large number of passport images accurately and efficiently.

V. USE CASE DIAGRAM

Preprocessing is the step of preparing and transforming raw data into a format that can be used for analysis or model training. It involves cleaning, transforming, and organizing data to remove noise, inconsistencies, and irrelevant information that can affect the quality and accuracy of the results. Preprocessing is a critical step in data analysis as it can improve the efficiency and effectiveness of data processing and model training. Common preprocessing techniques include data cleaning, data normalization, data scaling, feature selection, and feature engineering.

VII. PROPOSED METHODOLOGY

Scan the document, apply preprocessing and then extract its features .After that, segmentation helps in marking the image as different blocks of image textual and non textual data. Then lines and words are detected and segmented to get the character, which are further identified and recognized.

Methodology is divided into four subparts:

1. Image Preprocessin
2. Text localization and Extraction
3. Text/Non text classification
4. Character Recognition

VIII. RESULTS AND DISCUSSION

Passport recognition and information extraction technology has become an important tool for a variety of applications, including border control, identity verification, and travel management. The accuracy of this technology has greatly improved in recent years, with modern systems achieving near-perfect accuracy rates.

One important use case for passport recognition and information extraction technology is at border control checkpoints. By automating the process of verifying passport information, these systems can greatly reduce wait times and improve the efficiency of border control procedures. Additionally, by automating the process of data extraction from passports, errors can be reduced and the risk of fraud can be minimized.

Another important application of passport recognition and information extraction technology is in identity verification for online services. By using OCR and other methods to extract data from passports, online services can verify the identity of users with a high degree of accuracy. This can be particularly important for services that require high levels of security, such as financial institutions or government agencies.

Overall, the accuracy of passport recognition and information extraction technology has greatly improved in recent years, making it an important tool for a variety of applications. By automating the process of passport verification and data extraction, this technology can greatly improve the efficiency and security of a wide range of processes.

Conclusion

This is the discussion about optical character recognition techniques to translate the text from unknown language text into known language. The system has the capability to recognize characters with accuracy exceeding 90% mark. The advantage of this system is that it is easily portable and its scalability which can recognize various languages and also help in translating the text in different languages. The accurate recognition is directly depending on the nature of the material to be read and by its quality. Some changes we would like to bring in our project which includes improving the accuracy of classification model, extending the model to get dynamic feed method to capture live photo and recognize text in it. This way we can make our text recognition model more effective and useful for several purposes. Optical character recognition is most successful for constrained material, that is documents produced under some control. The applications for future OCR systems lie in the recognition of documents where control over the production process is impossible.

References

[1] R. G. Casey & K. Y. Wong. Document-Analysis Systems and Techniques. Image AnalysisApplications, eds: R. Kasturi& M. Tivedi, p. 1-36. [2] Karthick, K & Chitra S 0 7 ‘ovel Method for Energy Consumption Billing Using Optical Character Recognition’ Energ Engineering: Journal of Association of Energy Engineers, vol. 114, no. 3, pp. 64-76, ISSN:1998595. [3] K. Gaurav and Bhatia P. K., “Analytical Review of Pre-processing Techniques for Offline Handwritten Character Recognition”, 2nd International Conference on Emerging Trends in Engineering & Management, ICETEM, 2013. [4] Bagadkar S. L.; and Malik L. G., “Review on Extraction Techniques for Images, Text Lines and Keywords from Document Images,” IEEE International Conference on Computational and Computing Research, pp. 1-3, 2014. [5] Choudhary A.; Rishi R.; and Ahlawat S., “A New Approach to Detect and Extract [6] Characters from Off-Line Printed Images and Text,” First Elsevier International Conference on Information Technology and Quantitative Management, pp. 434 – 440, 2013. [7] Samadiani N.; and Hassanpour H. “A neural network-based approach for recognizing multi-font printed English characters,” Journal of Electrical Systems and Information Technology, vol. 2, issue 2, pp. 207-218, 2015. [8] THE PASSPORTS ACT, 1967 http://passport.gov.in/passport_act.pdf [9] Yang, Jufeng, Kai Wang, Jiaofeng Li, Jiao Jiao, and Jing Xu, “A fast adaptive binarization method for complex sceneimages,” 19th IEEE International Conference on Image Processing (ICIP), 2012. [10] Chu, Yung-Long, Hung-En Hsieh, Wen-Hsiung Lin, Hui-Ju Chen, and Chien-Hsing Chou. \"Chinese FingerReader: a wearable device to explore Chinese printed text.\" In ACM SIGGRAPH 2017 Posters, p. 54. ACM, 2017. [11] Kuruvilla, Jiss, Dhanya Sukumaran, Anjali Sankar, and Siji P. Joy. \"A review on image processing and image segmentation.\" In 2016 international conference on data mining and advanced computing (SAPIENCE), pp. 198-203. IEEE, 2016. [12] Shanmugam, K., and B. Vanathi. \"Hardcopy Text Recognition and Vocalization for Visually Impaired and Illiterates in Bilingual Language.\" In Computational Intelligence and Sustainable Systems, pp. 151-163. Springer, Cham, 2019. [13] https://www.cis.uni-muenchen.de/~fraser/information_extraction_2018_lecture/sarawagi.pdf

Copyright

Copyright © 2023 Meghana H M, Nagarathna P, Meghana S, Vikas H S, Krupashankari S.S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.